Field of the Invention
Embodiments of the present invention generally relate to using opportunistically placing structured light in projected frames.
Description of the Related Art
Hand-held projectors, such as the DLP (Digital Light Processing) Pico Projector available from Texas Instruments, are gaining in popularity as such projectors are increasingly becoming available in light-weight portable consumer electronic devices such as smart phones and cameras. The projector on these devices may be used as a display mechanism to project digital content onto any suitable surface.
In such hand-held projectors, the position and orientation of the projector may change during projection. Thus, tracking the relative surface depth and orientation and adapting the projected images dynamically without interrupting the video projection is desirable. For example, perspective distortion of the display images may result when the projector is not held perfectly perpendicular to the projection surface. This distortion is sometimes referred to as keystone distortion. A telltale artifact of keystone distortion is that a rectangle in the projector plane appears as a non-right-angled quadrilateral on the projection surface. If the orientation of the surface is known, the video content can be pre-warped on the projector's plane such that when projected onto the surface, the content appears to be aligned (or rectified) to the viewer.
Structured light can be used to measure depth by projecting a known pattern onto the world. The reflection of the pattern is captured by a camera and the depth profile of the environment can be computed based on triangulation between the known projected pattern and the image of the distorted pattern as captured by the camera. In the case of a light projector, the depth and orientation of the projection surface relative to the projector can be used to manipulate the projection content to satisfy various criteria. Examples include ensuring that the content is keystone corrected, remains in focus, or is of a constant size.
A commonly used method of applying structured light when projecting video is to replace several frames in the video with structured light frames (i.e., to steal frames). The projector and camera are synchronized such that the camera knows when to expect to see the pattern of the structured light. In addition, by designing pairs of structured light patterns that are the inverse of each other, the structured light can be rendered “invisible” to human observers. One drawback of replacing or stealing frames from the projected video for structured light is that the brightness or darkness of the projected video is reduced, which may deteriorate the viewing experience. For instance, a white-dark frame pair will add a constant grey brightness level to the video content, thus reducing contrast. In addition, in order to work well, this approach requires a projector and camera capable of operating at high speeds and in perfect synchronicity.